1. Field of the Invention
The present invention relates generally to video coding and compression, and more particularly to providing a method of improving the frame rate or picture size of a compressed video sequence beyond that which a given decoder would otherwise be able to process.
2. Description of Related Art
Digitization of video has become increasingly important. Digitization of video in communication (e.g., videoconferencing) and for digital video recording has become increasingly common. In these applications, video is transmitted across telecommunication links such as telephone lines, computer networks, and radio, or stored on various media such as DVDs, hard disks, and SVCDs.
Presently, efficient transmission and/or storage of video data requires encoding and compression of video data. Video compression coding is a technique of processing and encoding digital video data such that less information (typically measured in bits) is required to represent a good-quality rendition of the video sequence. Most video compression/decompression systems are “lossy”, which means that the reproduced video output is only an approximate, and not exact, version of the original input video data. Various compression/decompression schemes are used to compress video sequences.
Several approaches and standards for encoding and compressing source video signals exist. These include ITU-T Recommendations H.120, H.261, H.263, and H.264 (hereafter “H.120”, “H.261”, “H.262”, “H.263” and “H.264”, respectively), and standards promulgated by the International Organization for Standards/International Electrotechnical Commission (“ISO/IEC”) “MPEG-1” (ISO/IEC 11172-2:1993), “MPEG-2” (ISO/IEC 13818-2:2000/ITU-T Recommendation H.262), and “MPEG-4” (ISO/IEC 14496-2:2001). Each of these standards is incorporated by reference in its entirety.
In the most commonly used video compression systems, each uncompressed picture is represented by a rectangular array of pixels. In many operations, the whole image is not processed at one time, but is divided into rectangular groups of “macroblocks” (e.g., 16×16 pixels in each macroblock) that are individually processed. Each macroblock may represent either luminance or “luma” pixels or chrominance or “chroma” pixels, or some combination of both. A number of methods and techniques for macroblock-based processing of images are generally known to those skilled in the art, and thus are not repeated here in detail. All lossy video compression systems face a tradeoff between the fidelity of the of the decompressed video compared to the original and the number of bits used to represent the compressed video, all other factors being equal. For a given video sequence, different video quality may be produced by a video encoder for a fixed number of bits, if different compression techniques are used. Which techniques may be used, and their effectiveness, are in some cases dependent on the amount of computation, memory, and latency margin available to the compression system.
Existing methods of frame rate regulation allow an encoder to decrease (i.e., slow down) the frame rate to ensure that the image compression maintains an acceptable visual quality level, given other constraints such as computation, memory, and latency limits. Particularly, if enough of the image is changing, (meaning that a relatively larger number of bits will be required to maintain image quality), the encoder can slow down the frame rate to increase the available bits per frame. The prior art discloses a variety of existing methods that allow an encoder to run at a variable bitrate or variable frame rate. The prior art also discloses the idea of “skipping” coding for unchanged areas of the image. This technique has been used before for the purpose of reducing the bitrate of a video stream, or increasing the image quality (by increasing the bitrate of the coded areas to take advantage of the bits saved by not coding the unchanged areas).
The rate at which a video compression system (in both the encoder and decoder) can process frames is limited by a number of factors, such as the input frame rate, the bitrate of the compressed video stream, and the amount of computation the compression system can perform in a given period of time. Usually, in cases where there are ample input frames and available bitrate, the computation limit becomes the dominant limit on frame rate.
What is needed in the art is a technique that allows an encoder to increase (i.e., speed up) the frame rate dynamically based on a computational model of the decoder. Instead, ITU-T and MPEG compression systems define a fixed frame rate ceiling for a given picture size, which is based on the assumption that all macroblocks in each frame are coded. Another aspect of what is needed in the art is to further take advantage of the lowered decoding computation requirements when “skipping” is used, allowing the encoder to encode a faster frame rate than would otherwise be possible. The present invention is directed to such a system.